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ABSTRACT 


An  algorithm  is  presented  for  the  general  (not  necessarily 
convex  or  concave)  quadratic  programming  problem  over  a 
linearly  constrained  set.  The  algorithm  is  finitely 
convergent  and  makes  use  of  a  convex  quadratic  programming 
method  as  a  subroutine  (like  the  quadratic  simplex  for  instance). 
The  basic  tool  for  this  method  is  a  facial  decomposition  for 


polyhedral  sets 


Section  1)  Introductory  Remarks 


The  term  quadratic  programming;  has  now  become  classical  both  in 
management  and  engineering  sciences;  it  refers  to  the  optimization  or  a 
quadratic  objective  function  over  a  polyhedral  set  P  in  the  n-ditnensional 
vector  space  .  Usualiy,  however,  all  that  is  meant  is  convex  quadratic 


programming,  i.e. 

solving  problems 

of 

either  type: 

a)  minimize 

f(x)  ,  subject 

to 

x  e  P 

(la) 

or  b)  maximize 

g(x)  ,  subject 

to 

x  e  P 

(lb) 

where  f  is  a  convex  function  , 
g  a  concave  function  , 

and  P  a  polyhedral  set  defined  by  say  A  x  <  b  with  A  an  m 
by  n  matrix. 

Convex  quadratic  programs  are  indeed  very  important  because  they  appear 
in  practical  applications  and  also  because  they  represent  a  useful 
approximation  of  more  general  convex  programs . 

There  are  other  cases  of  linearly  constrained  quadratic  programs, 
however ,  (which  also  correspond  to  practical  problems)  namely  the  opposite 
of  (1 )  cafed  here  concave  programming  (quadratic)  problems,  i.e. 


a) 

maximi ze 

f(x)  ,  subject  to  x 

5  P 

(2a) 

b) 

minimize 

g(x)  ,  subject  to  x 

e  P 

(2b) 

A 

third  class 

of  quadratic  program 

is  the  general  one 

fas  it  contains 

both  the  convex  and  concave  cases),  where  one  seeks  the  optimum  (maximum 
and/or  minimum)  of  a  general  quadratic  function  f(x)  (not  necessarily 
convex  or  concave)  over  the  polyhedral  set  P  . 
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Footnote 


Footnote 


Footnote 


This  paper  focuses  on  this  general  case,  presenting  an  algorithmic 
principle  based  on  some  simple  fundamental  properties  of  F.  several 
versions  of  such  algorithms  are  being  tested  in  order  to  find  appropri¬ 
ate  measures  of  efficiency  for  algorithms  of  this  type. 

Probably  because  it  does  not  possess  the  many  useful  properties  of 
its  convex  special  case,  relatively  little  attention  has  been  devoted  to 
the  general  case  in  the  literature With  no  attempt  to  pres  .c  a  complete 
bibliography  of  the  subject,  let  us  mention  the.  articles  f  9  ,  10  ] 

by  Ritter  who  is  apparently  the  first  to  have  studied  the  non-convex 

case,  followed  by  Cottle  and  Mylander  [5]  and  more  recently  Konno  [7].  All 

2/ 

of  these  papers—  are  centered  around  the  concept  of  cutting  planes  and  [7] 
presents  an  interesting  study  of  several  types  of  cutting  planes  applicable  to 

non-convex  quadratic  programming,  including  the  cuts  introduced  by  Hoang 
Tui  in  [11]  for  general  concave  programming  and  which  have  recently 
received  much  attention  in  integer  programming  [ 1,3, 6, 8  ]. 

Our  approach  proceeds  along  different  lines,  however,  as  it  is  funda¬ 
mentally  oriented  towards  an  enumerative  decomposition  into  subproblems 

3/ 

of  the  original  problem.—  It  makes  use  of  the  so-called  facial  decomposi¬ 
tion  of  the  polyhedral  set  P  ,  which  leads  to  the  construction  of  an 
arborescence  a  I ong  which  one  may  search  for  the  global  optimum;  much  as 
for  the  branch  and  bound  algorithms,  the  arborescence  is  used  here  to 
isolate  "candidates"  (usually  local  optima)  for  the  optimum  which  then 
can  be  enumerated  to  identify  the  over-all  best  solution(s),  i.e.  the 
global  optimum(s). 

In  section  2  we  list  some  useful  properties  of  the  linearly  constrained 
general  quadratic  programming  problems.  Section  3  then  contains  brief 
summary  of  the  facial  decomposition  procedure  [  4  ]  and  section  a  outlines 


sow*  ro1o1fflrt  mathematical  daveInnTnKntS  . 
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Section  2 


Footnote 


)  The  structure  of  quadratic  programs 

We  present  below  a  list  of  the  properties  which  lie  at  the  crux  of 
the  problem  under  consideration;  all  are  straightforward  observations 
and  do  not  require  much  mathematical  insight.  For  simplicity  we  only  con¬ 
sider  a  bounded  n-ditnensional  polyhedron  P  as  feasible  region  and  the 
general  quadratic  problem  (GQP)  is  defined  by: 

minimize  f(x)  ,  subject  to  x  e  P 
where  f  is  a  quadratic  function. 

Call  S  the  set  of  all  optimal  solutions  x  of  GQP  . 


Theorem  1: 


Corollary 


Theorem  2: 


Theorem  3 : 


Corollary 


Theorem  4: 


If  SC  rel  int  (P)  then  f  is  strictly  convex  (i.e., 

positive  definite)  on  Aff(P)  and  S  consists  of  a 

unique  point  x  . 

4/ 

1.1: —  If  f  is  not  strictly  convex  on  P  then  f  attains 
its  global  minimum  on  the  relative  boundary  of  P  . 

The  relative  boundary  of  an  n-dimensional  polyhedron  P 
is  the  union  of  its  (n-1) -dimensional  (closed)  facets. 

If  f  is  convex  (concave)  on  P  then  it  is  convex  (con¬ 
cave)  on  all  the  facets,  and  lower-dimensional  faces  F 
of  P  . 

3.1:  Since  all  faces  of  P  are  polyhedral,  theorem  3  also 
hold  s  for  any  face  of  P. 

If  x  e  P  then  x  is  either  a  vertex  of  P  or  x 


belongs  to  the  relative  interior  of  some  face  of  P 
(including  P  itself). 
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Corollary  4.1:  Any  optimal  solution  x  of  GQP  satisfies  theorem  4. 
Theorem  5:  Let  c  ^  P  be  faces  of  P  ,  and  let  x  e  be 

optimal,  i.e. 


f(x)  <  f(x)  ,  V  x  e  F  . 


If 

x  e  rel  int 

(Ft) 

then 

f(x)  <  f(x) 

,  v 

X  e  F  £ 

Theorem  6: 

If  f 

is  concave 

on  a 

face  F  c.  p  then  there  exists  a 

vertex  x  of  F 

such 

that 

f(x>  < 

f  (x) 

,  V  x  e  F 

Proofs :  To  1:  Along  any  1 -dimensional  line,  the  quadratic  function  f 
is  either  strictly  concave,  strictly  convex  or  linear. 

Hence,  for  every  point  x  e  S  and  along  any  line  L(x) 
through  ^  ,  f  roust  be  convex  or  linear  since 
f(x)  <  f(x)  ,  V  x  e  L(x)  ',  but  suppose  that  f  is 
a  linear  function  along  every  line  L(x)  through  x; 
then  on  the  relative  boundary  of  P,  there  must  exist 
a  minimal  point  x  e  (L(x)  fl  S) ,  and  we  have  a  contradic¬ 
tion  to  the  hypothesis;  hence,  f  is  strictly  convex; 
furthermore,  a  strictly  convex  function  attains  a  unique 
•minimum  x  on  any  open  set  0(S)  S  for  instance  U  =  rel  int(P) 
hence,  S  =  {x} .  Q.E.D. 

To  1.1:  By  contraposition  of  theorem  1. 

To  2:  This  is  a  classical  result  and  the  proof  is  omitted. 


To  3: 


Immediate  since  F  c  p  . 


To  4: 


This  is  a  classical  result,  too.  The  polyhedron  P  can 


be  decomposed  into  disjoint  subsets  9^  =  rel  int  (F^) 
where  runs  over  all  faces  of  P  (including  P 

itself )  .'furthermore ,  since  the  relative  interior  of  a 
point  is  ill-defined,  one  has 

p  -Vv' 

where  F  are  the  vertices  of  P  and  F  the  k- 

P>  w 

dimensional  faces  of  P  ,  (1  <  k.  <  n)  . 

To  4.1:  Immediate  since  x  e  P  . 

To  5 :  Immediate  since  F2C  F^  but  F^  rel  int  (F^)  . 

To  6 :  Proof  omitted. 

The  above  properties  of  GQP  lead  in  a  straightforward  manner  to  the 
algorithmic  principle  presented  at  the  end  of  section  2. 

In  theorem  4  (its  proof  and  corollary)  one  finds  the  germ  of  a  con¬ 
struction  described  below  in  greater  detail. 


Consider  the  system 
Ax  —  b 


(3a) 


x  >  0  (3b) 

with  A  an  m  by  n  +  m  matrix  of  rank  m, 
and  set 

M  =  {l ,2, . . . ,n,n+l, . . . ,n+m}  ; 

A  subset  IC  M  is  called  minimal  if 

[x]xi  >  0,  Vi  e  I }CZ  fx|xj  >  0,  Vi  e  M}  *  P 
and  for  every  i^s  I,  there  exists  a  point  x  such  that 

x.  <0,  while  x  >  0,  Vi  e  1  -  {i } • 

1  0 

Geometrically,  it  may  be  seen  that  the  minimal  set  I  consists  of  the 

indices  i  e  1  for  which  the  affine  set  {x|xj  =0,  i  e  i]  contains 

at  least  one  facet  of  the  polyhedral  set  P  defined  by  (3).  Let 

I  be  the  minimal  set  of  P. 
o 

Note  that  a  minimal  set  Iq  can  be  found  by  solving  iteratively 
the  following  L.P.  starting  with  I  =  M: 

minimize  x^  ,  subject  to  >  0  ,  Vi  s  I  •  {iQ}  (3c) 

v 

Ax  -  b 

If  the  minimal  value  x-  is  <•  0  then  keep  i  in  the  set  I;  otherwise,  if 

1o  0 

X£0>  0,  then  reduce  the  set  I  by  eliminating  its  element  i^. 

It  is  shown  in  [4]  that  after  all  elements  i  e  M  have  been  considered 

o 

in  (3c),  the  remaining  set  1  =  1^  is  minimal. 

Now  the  facial  decomposition  of  an  n-dimensional  polyhedron  P 
simply  consist  in  determing  the  minimal  index  set  I  =  I(i^)  corresponding 
to  each  (n-l)-dimensional  facet  F(l^)  c  {x  c  Pjx^  =  0,  ij€  Iq) ,  the 
set  I  =  I(i^»i2)  ^or  each  (n-2) -dimensional  face 
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F(i1,i2)  =  {x  e  F(i1)C  P  |  x.  =  x±  =  0  ,  e  1(1^)  c  IQ]  , 

etc....;  each  index  set  I(i^,...,i  )  characterizes  a  face  F(i^,...,i  ) 


‘ (i^,...i  )  {_ 


xeFj  x  =  . . .  =  x  .  =  0  ,  Ax  =  b  ? 


x.  >  0  ,  Vi  e  I  (i,  , . . .  i  . 
l  —  i  q-l 


which  lies  in  the  subspace  Aff^F(i^,. . . ,i  defined 


x.  =  x,  —  ...  =  x .  =0  ; 

i.  i~  l 

12  q 


and,  of  course,  one  has 

F(i^)Z5  F  (i^  ,  i2)  ID  Ol(i^»...>i)  , 
t(lt)0  KVlj)3  ...  , 

and  D  *£f(F(l1,l2))3  ....  O Af ff F(i; . !  )j. 

The  complete  facial  decomposition  of  P  therefore  assumes  a  tree¬ 
like  structure  beginning  with  a  single  node  which  corresponcs  to  the.  largest 
face  P  (which  contains  all  the  other  faces);  from  this  node,  one  has 
branches  (as  many  as  there  are  elements  in  I  ) ,  each  one  leading  to  a 
facet  F(i)  ,  i  e  Iq  :  then,  on  the  third  level,  one  finds  the  (n-2)- 

dimensional  faces  of  P  ,  i.e.  the  facets  of  F (i) ,  i  e  I  ;  from  each 

o 

F(i^),  i^*  1q  one  therefore  has  branches  going  to  the  faces  F(i^,i2> 
with  i2s  I(i^)  e:  Iq  ;  and  so  on.. .the  lowest  level  containing  all  the 
0-dimens ional  faces  (vertices)  of  P  . 

Note  that  this  tree-structure  is  redundant  because  F(i,,...i  ) 
corresponds  to  one  and  the  same  face  fot  ail  permutations  of  the  indices 
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In  order  to  avoid  this  type  of  redundancy,  it  suffices 
to  generate  the  index  sets  . iq]  in  a  strictly  increasing  lexico¬ 

graphic  order.  A  more  detailed  description  of  the  facial  decomposition 
method  can  be  found  in  [4].  In  particular,  a  method  is  given  for 
eliminating , in  the  facial  arborescence,  another  kind  of  redundancy 
due  to  degeneracy.  Ultimately,  the  nodes  of  the  arborescence  will 
be  in  one-to-one  correspondence  with  the  faces  of  P. 

Algorithmic  principle:  The  face  decomposition  of  P  furnishes,  in  quite 

a  natural  way,  an  enumerative  method  to  solve 
linearly  constrained  quadratic  problems.  Consider  a  face  F  of  P 
generated  at  some  level  of  the  face  decomposition: 

A)  If  f  is  convex  on  F  ,  then  we  know  that  it  is  convex  on 

every  face  and  subface  of  F  (Theorem  3).  We  may  therefore  solve  the 
convex  quadratic  problem: 

minimize  f(x)  ,  subject  to  x  e  F  ; 

then  store  the  optimal  solucion  x(F)  ,  remembering  that 
it  is  a  candidate  for  the  global  optimum  o_f  the  original 

problem:  minimize  f(x)  ,  subject  to  x  c  P  . 

Furthermore,  since  x(F)  is  the  optimum  on  F  ,  there  is 
no  need  to  investigate  the  faces  and  subfaces  of  F  (Theorem  3) 
individually:  thus  the  branch  can  be  terminated  at  F  . 

B)  If  f  is  concave  on  F  ,  then  we  know  that  the  optimum 
of  the  concave  quadratic  problem: 

minimize  f(x)  ,  subject  to  x  e  F 

is  attained  on  (at  least)  one  vertex  of  F  (Theorem  6);  one  there  fore- 
may  proceed  with  the  race  decomposition°f  P ,  without 


checking  the  concavity  of  £  on  each  face  and  subface 
of  F  ,  until  the  lowest  level  (the  vertices)  is  reached. 

The  determination  of  the  optimal  vertex  x(F)  can  be  made 
by  (explicit  or  implicit)  search  in  this  set  of  vertices: 
again,  x(F)  is  stored  as  a  candidate  for  the  global 
optimum  of  the  original  problem.  Note  that  one  may  also 
use  here  any  other  concave  quadratic  programming  algorithm, 
such  as  the  ones  studied  in  {2]  and  [7]  for  instance. 

C)  The  third  remaining  case  is  that  where  f  is  neither 

concave  nor  convex  on  F  ;  here  one  simply  proceeds  with  the 
facial  decomposition  of  F  ,  generating  new  faces  which 
must  be  tested  for  convex-  or  concavity. 

Termination:  It  is  easily  seen  that  an  algorithm  based  on  A),  B),  and 

C)  will  terminate  in  a  finite  n'unber  of  steps;  indeed  the 
face  decomposition  generates  a  finite  number  of  polyhedral  sets  (faces). 
Thus  the  only  point  which  could  lead  to  an  infinite  sequence  of  numerical 
operations  is  A;;  however,  algorithms  for  convex  quadratic  problems  on 
a  polyhedral  set  F  (like  the  quadratic  simplex  algorithm)  are  con¬ 
vergent  in  a  finite  number  of  steps. 

Finally,  one  compares  all  the  candidates  x(F)  obtained  at  the 
termination  of  all  branches  and  selects  the  "best"  one(s)  as  the  global 
optimum(s) . 


Concluding  Remarks : 

The  above  algorithmic  principle  is  not  exempt  of  difficulties  inher¬ 
ent  to  the  nature  itself  of  the  general  quadratic  problem  over  a  pojyhedral 
set  P  : 

1)  It  may  happen  that  f  is  "essentially  concave"  on  P  ,  that 
is,  that  there  are  only  relatively  few  faces  F  of  dimension 
greater  than  zero  (i.e.,  other  than  vertices)  where  f  is  con¬ 
vex,  In  this  case,  the  efficiency  of  the  algorithm  is  limited 
by  the  same  phenomenon  as  for  the  concave  problems ,  namely  the 
large  number  of  faces  and  especially  vertices  of  P  (this 
number  grows  exponentially  with  the  dimension  of  P  ,  in 
general).  In  order  to  avoid  the  explicit  enumeration  of  all 
the  vertices  of  P  ,  one  has  to  use  lower  bound  (for  a  mini¬ 
mization  problem)  estimates  in  order  to  truncate  branches 
which  are  obviously  suboptimal;  this  approach  corresponds  to 
the  classical  branch  and  bound  method.  Its  efficiency 
critically  depends  on  the  ability  of  the  bound 

estimates  to  truncate  relatively  high  -dimensional  faces, 
while  requiring  only  a  non-prohibitively  large  amount  of  com¬ 
putations  to  obtain  the  bounding  value.  The  developments  here 
are  highly  heuristic  and  problem-dependent  and  therefore, 
outside  the  scope  of  th^ c  paper. 

2)  Another  disagreeable  phenomenon  is  due  to  degeneracy  in  the 
polyhedral  set  P  .  Indeed  it  may  happen  that  the  face 
decomposition  is  geometrically  redundant,  while  its 
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algebraic 
To  clarify 
F^  and  F 


charm-.  erizatior.  by  index  sets  [i,  ,...,i  }  is  not. 

i  q 

this  point,  let  us  simply  mention  that  two  faces 
-  with  non-identicaj  index  sets,  i,e. 


»  ^2^1*  •  * • »3r) 

with  (i1,...,i  ]  * 

may  be  identical , from  a  practical  point  of  view, in  that  their 
point  sets  are  the  same,  i.o. 


{x  j  x  e  F,]  =  {x  f  x  e  F..1  , 

Degeneracy  can  he  observed  in  the  construction  of  the  minimal 
sets  I  ,  however,  and  may  there.fore  be  eliminated  from  the 
tree-structure  by  appropriate  bookkeeping.  Since  this  is  not 
immediately  pertinent  to  the  quadratic  programming  aspect  but 
rather  stems  intrinsically  from  the  lace  decomposition  of  P  , 
interested  reader  is  referred  to  [4  ]  for  further  details. 


3)  For  the  case  where  the  dimension  of  P  is  not  prohibitively 
large  (say  less  than  30  to  100)  and  where  the  non-convexity 
of  £  is  not  dominant,  i.e.  where  f  is  convex  on  all  (but 
a  few)  low-dimensional  faces,  the  present  approach  seems  prom¬ 
ising  particularly  in  view  of  the  .  (herent  difficulty  of  this 
type  of  problem.  In  any  case,  it  is  not  difficult  to  find 
problems  where  the  approach  presented  here  is  clearly  superior 
to  cutting  plane  techniques,  simply  because  it  does  not  entail 
the  convergence  obstacles  inf  educed  by  the  latter. 


the 


4)  Relation  to  other  work. 

A  similar  approach  to  the  facial  decomposition  has  been  sketched  by 
Murty  in  (13].  He  shows  that  the  optimal  solution  can  be  found  in  a  finite 
list  of  optimal  solutions  to  convex  quadratic  subprcblems.  Hie  list  of  sub¬ 
problems  is  obtained  from  a  full  combinatorial  arborescence  of  affine  sets, 
defined  by 

x  =  x  =  ...  -  x.  =  0  ,  i  e  M 

h  l2  \ 

where  0  <  k  <  n  ; 

Finiteness  of  the  procedure  follows  because: 

n  k 

a)  There  are  at  most  A  =  £  CK  <  2n  distinct  affine  sets  in 

k-o  n4™  ~ 

the  arborescence; 

b)  Only  the  reduced  problems  which  are  of  the  convex  quadratic 
type  need  be  solved  (in  finitely  many  steps). 

In  Murty's  procedure  =  ir  +  (n  -  k)  new  subproblems  are  generated  in  the 
arborescence  for  every  affine  set  S(n^,  m^)  of  dimension  =  (n  -  k) 
with  properties 

(ii)  S(n^,  m^)  intersects  the  feasible  set, 

(iii)  The  objective  function  f  is  not  convex  on  S^,  n^) 

(Note  that  if  no  additional  bookkeeping  organization  is  implemented  this 
procedure  generates  the  same  affine  set  k!  times,  corresponding  to  the  per¬ 
mutations  of  i^,...,!^) 

The  facial  decomposition  algorithm  presented  in  section  4  generates  distinct 
subproblems  and  their  enumeration  is  curtailed  according  to  the  remarks  below 


1)  Only  faces  of  the  feasible  set  P  ,  i.e.  only  those  affine,  sets 


S(n^,m^)  which  are  known  a  priori  to  intersect  P  ,  are 
generated. 

2)  Because  the  convexity  tests  indicate  that  f  is  either  convex  , 
or  concave .or  neither  convex  nor  concave  on  the  particular 
face  S  under  consideration,  there  is  no  extra  cost  for  using 
this  information  (when  the  function  to  be  minimized  (maximized) 
on  S  turns  out  to  he  concave  (convex)).  Indeed ,  when  f  is 
concave  on  S  ,  we  know  a  priori  that  the  vertices  of  S  can 
be  generated  directly  with  no  additional  convexity  test  (see 
part  B)  of  the  algorithmic  principle  and  STEP  4  B  of  the  algo¬ 
rithm)  . 

Finally  let  us  note  that,  in  general,  the  above  remarks  1)  and  2)  may 
be  expected  to  generate, for  the  amount  of  computations  of  the  facial  Jecomposi 
tion  method, a  reduction  which  grows  exponentially  with  the  dimension  n  of 
the  problem  as  compared  to  the  approach  of  [13]. 
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4)  Computational  aspects. 

Consider  the  set  of  linear  constraints 

k'  x'  <  b 

where  x*  >  o 

is  an  m  by  O  matrix 
x'  an  n-vector  (x,  ,...,x  ) 

1  Q 

b  an  m-vector  (bnll,....bLlt J 


As  customary  in  linear  programming,  let  us  introduce  slack  variables 
*k 

n 

*1.  =*  b  -  r  a'  x'  ,  k  =  n+1 , . . .  ,n+m 

K  K  j-jJ  KJ  J 


System  (3)  then  becomes 


Ax  —  b 

(4a) 

X  >  o 

(4b) 

with 

and 


A  =  (A',  I  )  where  I 
m 

X  =  (x1,...,xn,xi^1,.. 


is  a  mby™ 


,Xn+m^ 


unit  matrix, 


Furthermore,  take  the  objective  function  f(x)  to  be  defined  by 

f(x')  =  x'TCx'  +  Dx'  +  f°  (3) 

where  C  is  a  n  by  n  symmetric  matrix 

D  an  n-vector 
f°  a  scalar. 

(This  is  the  general  form  for  a  quadratic  function  in  n  variables 
x'  .) 
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Substltuting  into  a  quadratic  form: 

A  face  F j  of  P  characterizes 
Xj ,  ]  e  J  which  are  kept  at  value  zero 


a  set  of  variables 


x .  =  o 
J 


V  j  e  J  c  { 1 ,2 ,  „ . .  ,n  ,n+l , . . .  ,n+ta}  =  M 


(6) 


and  we  need  represent  the  quadratic  function  £  in  the  affine  space  Aff(Fj) 
defined  by  (6).  Consider  a  basis  B  for  the  system  (4),  where  the  set 
J  is  contained  in  the  non-basic  set.  The  general  solution  to  (4a)  can 
be  described  by 

=  B"Xb  -  (7) 


where  B  is  a  set  of  rr  linearly  independent  coiums  of  A  ,  forming  a 
non-singular  m  by  m  matrix  (basis); 

and  N  contains  the  remaining  columns  of  A  ;  the  variables  are 

called  "basic”  and  "non-basic.” 

A  general  point  in  the  affine  space  Aff  (Fj)  defined  by  (6)  then 

reads 


*B 


=  B~1b 


B 


(8a) 


arbitrary 


(8b) 


—  o 


(8c) 


where  the  matrix  is  obtained  from  the  submatrix  N  by  deleting  the 

columns  corresponding  to  j  e  J  . 

Let  us  substitute  (8)  into  (5)  in  order  to  obtain  the  expression  for 
the  function  £  ,  restricted  to  a  smaller  domain  of  definition,  namely 


Aff  (Fj) 
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Since  only  the  original  variables  x/  =  (x,  ,x? , . . .  ,x^)  are  present 
in  the  argument  of  the  function  £  ,  we  only  need  substitute  for  the 
variables  x*  >  j  e  [l,2,...,n]  and  basic,  i.e. 


\  -  <E“Vb  -  (B~1)"NJ  x^  (9) 

=  3b  -  0NT  ,  where  3  denotes  the  matrix  (B  *)' 

obtained  from  (B  "*")  by  deleting  the  rows  corresponding  to  basic  variables 
j  e  {n+1 , . . . ,n+m}  . 

Let  us  now  substitute  (9)  into  (5),  yielding 


£  =  i  3b  -  gN  x  I  C  !  3b  -  3Nx  i  + 
J  Njj  e  J  HjJ 


+  ]  4  e° 


-=  (f°  +  bTb1cgb  +  ogh)  + 


+  (-DgNjX^,  -  bTg "^CgKjX^  -  xj  NjgTCgb) 
1 J  J  J 

m  rp  rp 

+  \/jB  C@NJ5tKJ 


/p  rp 

xZ  N:C,N,x  +  L)  N  x  +  c° 
Nj  J  J  J  Kj  J  J  N  J 


(10a) 


where 


C^.  =  p‘  C  g  ,  which  is  (again)  a 


ic  matrix 


(10b) 


Dj  =  (03  +  2  b1  Cj) 


fJ  “  f°  +  °Jh  +  ,jT  CJ6 


(10c) 


(10d) 
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Thus,  one  obtains  for  f,  restricted  to  Aff  (Fj)  »  a  function 
fj(xN  )  which  is  again  a  (general)  quadratic  function. 

Testing  the  convexity  of  a  quadratic  form. 

It  was  indicated  in  section  3  that  one  of  the  major  ingredients  of 
the  algorithmic  principle  developed  there  is  the  test  for  convexity  (and 
concavity)  of  the  given  quadratic  objective  function  on  a  subset  F^  of 
its  domain  of  definition.  To  be  precise,  we  must  repeatedly  explore  the 
definiteness  of  f  fx)  >  for  x  e  F*^  P  ,  i.e.  on  a  face  F_  of  P  . 

.  j  -  j  j 

According  to  the  above  result  (10),  this  task  can  be  seen  to  consist 
in  finding  the  definiteness  of  the  quadratic  form  in  the  (n-J)  variables 


fJ(XN>  *  £J  +  "j  %  *  4,  "I  CJ  NJ  X  <10> 

Jo  J  J  J 

One  has : 

T 

fj  is  convex  if  the  symmetric  matrix  CT  is  positive 
(semi-)  definite; 

hence,  it  is  sufficient  to  investigate  the  definiteness  of  the  symmetric 
matrix 

C=nJ  8T  CpN  (10a) 

Ihere  are  many  theoretical  and  practical  approaches,  concievable  at  this 
point  and  we  present  but  one  method  below,  chosen  because  it  seems 
adequate : 
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I 


Consider  the  quadratic  function 


g(x)  -  x  C  x 


where  C  is  a  symmetric  k  by  k  matrix  ;  setting 


y  =  Cx  ,  or  yt*  £  cljXj  ,  i  -  1.2 . * 

g(x)  becomes  the  scalar  product  xy  of  the  k-vectors  x  and  y  : 


g(x)  **  xy  =  T  x  y  . 

j-1  J  J 

Now,  by  pivoting  in  a  non-zero  diagonal  element  of  C  (say  c  )  one 
can  express  a  variable  x.  (here  x^)  in  terms  of  the  (k-1)  remaining 
x  variables  and  one  y  variable  (here  y  )  ;  this  operation  is  known 
as  a  gaussian  exchange( principal  pivoting)in  the  linear  system  (12). 

Wore  concretely,  one  has  after  the  gaussian  exchange  with  pivot 

~  (*0) 

11 


i/  ^  \  ,  k  c,  . 

-  (  -y.  +  2  c  x .  j  -  - —  y .+  7  — ■*- 

cn  J-2  J  cn  l-z  cu 


(13a) 


“d  for  “  2-’k:y‘  ■  X  Vi +  '  >  (-v 


l  (-.j  -  ) *, 


(13b) 


Let  us  now  replace  and  y^  b7  their  respective  expressions 

(13)  in  g(x)  , 


Vi  *  -(r-  yi +  ji  r1  *i)v-  .L^rV  it 


k  r-C. 


/w 

c  c 
Slci1> 


s=-2  cu  js2  cu 


12  r  £  c  ii  ~ 
~  yi  +  yi  L‘  ,r2  ?  *j  + 

CL1  J  1  11 


c  - 
si  ~ 

i  k 

1  4-  V 

U  -  sl  li' 

c  s- 

C11 

1=2 

Vejs  ~  , 

C11 

~  yj  +  £  c  x  x 

~  1  j®  S  J 


since  c.  “  c  ,  for  J»i  by  symnetry  of  C  . 
lj  81 


Furthermore,  by  construction,  the  matrix  C  is  again  symmetric; 


c  . c.  .  c  .,c, 

-  sill  ~  Ills  “ 

C  .  *=  C ,  -  —  *c  .  “  1  ■  =  C 


JS  is  ~ 


Now  since  >  0  ,  one  has  the  necessary  condition: 


g  positive  definite  *®  c^^  >  0  , 
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and  similarly 

g  negative  definite  •»  <  0  . 

The  test  for  strict  convexity  of  the  given  (general)  quadratic  func¬ 
tion  f  ,  on  a  restricted  domain  of  definition  (Aff  Fj  )  can  be  made  as 
follows : 

Step  1:  Check  that  all  elements  c,.  of  C„  in  the  diagonal  are 

XX  J 

positive;  if  this  is  not  the  case  then  g  is  not  strictly 
convex;  stop 

Step  2:  Pivot  on  and  generate  the  new  symmetric  matrix  Cj  ;  note 

that  the  number  of  rows  and  columns  of  C  is  one  less  than 
for  Cj  ;  go  to  Step  1. 

In  at  most  (n-J)  iterations  this  procedure  shows  that  Cj  is  positive 
definite  or  not. 

Remarks :  Since  the  gaussian  elimination  is  always  applied  to  3  symmetric 
matrix,  yielding  a  new  matrix  which  is  again  symmetric,  explicit 
computations  need  only  he  carried  in  the  upper  triangular  part 
of  C  ;  this  represents  a  substantial  reduction  in  the  number 
of  necessary  operations.  In  fact  principal  pivoting  is 
a  standard  manipulation  in  tnatheraa?  ‘cal  programming,  which 
can  be  done  efficiently. 

Mote  that  in  Step  2,  the  choice  of  c^  was  arbitrary;  in  fact 
an  efficient  algorithm  will  try  to  exhibit  non-convexity  as  soon 
as  possible;  thus  one  will  choose  the  Index  i  of  the  principal 
pivot  c _  ,  with  greater  care. 
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Outline  of  the  algorithm: 

We  may  now  summarize  the  various  results  obtained  in  the  previous 
sections  into  an  algorithm  to  solve  general  quadratic,  linearly  con¬ 
strained  problems.  ftie  algorithm  described  below  is  but  one  version  of 
the  principle  presented  at  the  end  of  section  3.  Since  numerical  compu¬ 
tations  are  still  at  an  experimental  stage,  it  is  difficult  to  assess 
the  computational  efficiency  of  one  version  with  respect  to  another, 
especially  when  the  differences  3tem  from  programming  details  rather  than 
basically  different  approaches.  The  algorithm  below  was  chosen  because 
of  its  relative  simplicity  of  exposition  combined  with  a  robust  numerical 
performance. 

The  algorithm  requires  the  following  quantities: 

two  (n+l)-arrays  :  m  ,  [0:n]  and  (trt-1)  "dynamic"  arrays  (at  roost 

(n+m)  dimensional)  :  I[o;  ]  ,  1(1;  ]  n;  ]  . 

The  result  is  found  in  the  (n+m) -array  XqpT  with  the  optimal  value 
for  f  denoted  by  OPT  . 

STEP  0: 

Let  t  :  iteration  index  (level  in  the  arborescence) 
m  :  pointer 
I  :  minimal  index  set 

number  of  elements  in  I 
OPT:  current optimal  value  for  f  over  P  . 
initialization:  t  :  °  1  :  mfOl  :  «  1  :  1(0:  1  :  *  1  =  minimal  set  of  P' 

■ '  O  ' 

“j  =  number  of  elements  in  Iq  ;  OPT  =  +  ®  (minimization) 
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STEP  1; 

-  Test  the  definiteness  of  f  on  the  affine  set:  Aff(F  ^  : 

Aff  (Ft-1)  *  {x  j  Xj  *  0  ,  for  ail  j  *=  lj^0;m[0] ], . . .  t-1]  j  ; 

-If  £  is  convex  then  go  to  STEP  4a  ; 

-If  f  is  concave  then  go  to  STEP  4b  ; 

STEP  2: 

-  Find  the  minimal  index  set  I{t;  ]  of  the  polyhedral  set  F^ 

r  (A, I)  x  =  b 

_  x  >  0 

Ft-l  : 

-  0  ,  for  all  J  *  l[^0;m[0j  j,... 

I 

^  ...,l[t-l;m[t-lj] 

-  Denote  ly  the  number  of  elements  in  I[t;  ]  ; 

STEP  3: 

-  Let  m{t]  .  “  l  J 

t  :  *  t  +  1  ; 

-  go  to  STEP  1  ; 

STEP  4a: 

-  Solve  the  '.onvex  quadratic  problem: 

minimize  f(x)  ,  subject  to  x  c  F  ^ 

(using  one  of  the  quadratic  simplex  algorithms,  for  instance). 


-  go  to  STEP  5 
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STEP  4b: 

Solve  the  concave  quadratic  problem 

minimize  f(x)  ,  subject  to  x  e  ^ 

(This  can  be  done  by  determination  of  the  vertices  of  ,for 

instance;  these  vertices  are  obtained  by  iteratively  computing  STEP  2 
and  STEP  3,  bypassing  STEP  1,  until  t  ■=  n  where  the  O-dimensional  faces 
are  generated  [4]). 

STEP  5: 

Denote  by  f  and  x  the  optimal  solution  obtained  in  STEP  4. 

If  f  <  OPT  then  set  x0p.p  '  —  x 

OPT  :  =  £ 

STEP  6: 

If  m{  t-1]  <  then 

set  m[t-l]  ;  =  m{  t-1]  +  1  and  go  to  STEP  1. 

Otherwise 

set  t  :  *»  t-1  ; 

if  t  >  0  then  go  to  STEP  6 

if  t  =  0  then  {sTOP"|  ;  the  global  optimum  is  x0?T  ,  with  value  OPT  . 

The  algorithm  can  be  seen  to  contain  the  following  components; 

Branch- identification  phase  (STEP  2)  ;  Given  a  (n-t) -dimensional  face 
of  the  n-d iinens ionai  polyhedron  P  ,  the  minimal  set 
I[t;  ]  is  constructed,  which  identifies  all  the  (n-t-1)- 
dimensicmal  facets  of  F  ^  . 


Convexity  test  phase  (STEP  1)  :  Given  a  subspace  Aff  (F  one  calculate, 

the  restricted  function  f  >  and,  by  principal  pivoting, 
finds  out  if  f  is  convex,  concave  or  neither. 

Convex  (01  concave)  programming  pnase  (STEP  4)  :  Knowing  that  the  pro¬ 
gramming  subproblem 

minimize  f  , (x)  ,  subject  to  x  e  F 

t-1  J  t-1 

is  either  convex  or  concave,  one  finds  the  optimal  solution (s ) , 
by  applying  the  respective  algorithm. 

Ordering  and  choice  routines  :  Each  array  l[t;  ]  is  constructed  and 

arranged  in  an  arbitrary  but  fixed  way  and  the  choice  in  STEP  6 
amounts  to  a  well-defined  but  arbitrary  order  in  which  the 
faces  of  P  are  generated  and  examined. 

In  fact,  the  arborescence  constructed  by  the  algorithm 
is  guided  by  the  following  two  rules: 

Forward  choice  ruL  ("STEP  3):  Given  a  (n-t) -dimensional 
affine  space  Aff  (F  ^ )  for  which  the  convexity 
test  has  indicated  that  f  ^  is  neither  convex  nor 
concave ,  STEP  3  chooses  one  (the  "first"  one) 
(n-t-l)-dimensional  affine  space  which  is  a  subspace 
of  Aff  (F  ;  clearly,  because  of  the  arbitrary 

order  in  I[t;  ]  ,  this  choice  is  well-determined 
but  arbitrary. 

Backtrack-choice  rule  (STEP  6):  Given  a  (n-t)-dimensional 


affine  space  Aff  (F^^)  where  the  function  f  (i.e.  ) 


is  cither  curve. 


lit  r.ui'icava .  wa  (-now  that  the  cur  re  * 
branch  can  be  f.orc.ir.uted  because  the  optimal  solution  cor- 
responding  to  that  affine,  space  is  found  in  STEP  4;  here 
STEP  6  chooses  the  nett  cel lateral  s n~t ; -dimensional 
aff  ine  space,  :n>.  c  v-ing  the  pointer  mf  r.-l]  by  one; 
if  there  is  no  such  set  (i.e.  tuft  '_]  ^  m^lt-l])  ,  STEP 
6  backtracks  tc  the  nrevj  ous  level,  reducing  t  by  one, 
and  chooses  the  next  collateral  (n-t-l)-dimensional  affine 
space.  Here  again,  the  sequence  in  which  affine  spaces 
are  chosen  is  determined  by  the  order  within  the 
index  sets  Ift-1;  ]  and  If  t — 2 ;  J  ,  and  it  is  therefore 
well-defined  but  arbitrary. 

The  reader  familiar  with  branch  and  bound  procedures  will  recognize 
here  the  immediate  possibility  (or  eventual  improvements  of  the  overall 
efficiency  by  making  use  of  a  more  elaborate  choice  rule  which  selects 
every  element  of  the  index  sets  lit,  ]  e^acr.i\  once  bur  in  a  certain 
order  of  preference  defined  by  additional  computations .  To  be  efficient 
this  technique  must  be  combined  with  the  bound  estimates  mentioned  in 
section  3  and  no  effort  is  macie  to  go  into  the  art  involved  by  such  an 


approach 


I  t.  s 


'An  ir.cercs  ting  turv.'.,'  artio  ;c  j  i4i  on  qua;-  and  pseudo -convex 

by  W.  C.  My  lander  i  :>  up;  .r  snor* ly  in  c-r>«rat.  inns  He  anarch ;  these 
are  special  cases  oi  rior.-couve  -  quadra i  ic.  nrograuar.i  ng  which  can  still 
be  solved  in  eery  muon  the  ;ar:c  way  as  quadra*.!  v  programs  by 

some  algorithms.  These  o.uv*  way  nos.n  ■  s  u:n.  r...  ;i.  i  -'fa  eigenvalue, 

while  the  general  case  consider  here  may  have  arbitrary  eigenvalues 
(  >  0  ,  --  0  ,  ct  '  *)  *  |l-!j  ,  Myi  nr.de  r  report.-.  an  extension  oi 

Lemke  1  s  algorithm  to  .die  is  c-nn  cave. 


—  In  [12],  Mueller  pro  rents  :•  rando,.-.;;.  genera  to  J  heuristic  approach, 
making  use  ci  properties  similar  to  those  presented  in  section  2. 


—  A  closely  related  approach,  has  teen  suggested  in  j  131  and  is  briefly 
compared  with  the  present  one  at  the  end  of  section  3 . 


—  A  similar  result  lias  been  prove*,  by  Mueller  i.n  j  12] 
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